This text provides a critique of the subjective Bayesian view of statistical inference, and proposes the author's own error-statistical approach as an alternative framework for the epistemology of experiment. It seeks to address the needs of researchers who work with statistical analysis.

Algorithms are now routinely used in decision-making; they are potent components in decisions that affect the lives of individuals and the activities of public and private institutions. Although use of algorithms has many benefits, a number of problems have been identified with their use in certain domains, most notably in domains where safety and fairness are important. Awareness of these problems has generated public discourse calling for algorithmic accountability. However, the current discourse focuses largely on algorithms and their opacity. I (...) argue that this reflects a narrow and inadequate understanding of accountability. I sketch an account of accountability that takes accountability to be a social practice constituted by actors, forums, shared beliefs and norms, performativity, and sanctions, and aimed at putting constraints on the exercise of power. On this account, algorithmic accountability is not yet constituted; it is in the making. The account brings to light a set of questions that must be addressed to establish it. (shrink)

Scholarship on embedding values in AI is growing. In what follows, we distinguish two concepts of AI and argue that neither is amenable to values being ‘embedded’. If we think of AI as computational artifacts, then values and AI cannot be added together because they are ontologically distinct. If we think of AI as sociotechnical systems, then components of values and AI are in the same ontologic category—they are both social. However, even here thinking about the relationship as one of (...) ‘embedding’ is a mischaracterization. The relationship between values and AI is best understood as a dimension of the relationship between technology and society, a relationship that can be theorized in multiple ways. The literature in this area is consistent in showing that technology and society are co-productive. Within the co-production framework, the relationship between values and AI is shown to be generative of new meaning. This stands in stark contrast to the framework of ‘embedding’ values which frames values as fixed things that can be inserted into technological artifacts. (shrink)

Theories of statistical testing may be seen as attempts to provide systematic means for evaluating scientific conjectures on the basis of incomplete or inaccurate observational data. The Neyman-Pearson Theory of Testing (NPT) has purported to provide an objective means for testing statistical hypotheses corresponding to scientific claims. Despite their widespread use in science, methods of NPT have themselves been accused of failing to be objective; and the purported objectivity of scientific claims based upon NPT has been called into question. The (...) purpose of this paper is first to clarify this question by examining the conceptions of (I) the function served by NPT in science, and (II) the requirements of an objective theory of statistics upon which attacks on NPT's objectivity are based. Our grounds for rejecting these conceptions suggest altered conceptions of (I) and (II) that might avoid such attacks. Second, we propose a reformulation of NPT, denoted by NPT*, based on these altered conceptions, and argue that it provides an objective theory of statistics. The crux of our argument is that by being able to objectively control error frequencies NPT* is able to objectively evaluate what has or has not been learned from the result of a statistical test. (shrink)

Despite the widespread use of key concepts of the Neyman–Pearson (N–P) statistical paradigm—type I and II errors, significance levels, power, confidence levels—they have been the subject of philosophical controversy and debate for over 60 years. Both current and long-standing problems of N–P tests stem from unclarity and confusion, even among N–P adherents, as to how a test's (pre-data) error probabilities are to be used for (post-data) inductive inference as opposed to inductive behavior. We argue that the relevance of error probabilities (...) is to ensure that only statistical hypotheses that have passed severe or probative tests are inferred from the data. The severity criterion supplies a meta-statistical principle for evaluating proposed statistical inferences, avoiding classic fallacies from tests that are overly sensitive, as well as those not sensitive enough to particular errors and discrepancies. Introduction and overview 1.1 Behavioristic and inferential rationales for Neyman–Pearson (N–P) tests 1.2 Severity rationale: induction as severe testing 1.3 Severity as a meta-statistical concept: three required restrictions on the N–P paradigm Error statistical tests from the severity perspective 2.1 N–P test T(): type I, II error probabilities and power 2.2 Specifying test T() using p-values Neyman's post-data use of power 3.1 Neyman: does failure to reject H warrant confirming H? Severe testing as a basic concept for an adequate post-data inference 4.1 The severity interpretation of acceptance (SIA) for test T() 4.2 The fallacy of acceptance (i.e., an insignificant difference): Ms Rosy 4.3 Severity and power Fallacy of rejection: statistical vs. substantive significance 5.1 Taking a rejection of H0 as evidence for a substantive claim or theory 5.2 A statistically significant difference from H0 may fail to indicate a substantively important magnitude 5.3 Principle for the severity interpretation of a rejection (SIR) 5.4 Comparing significant results with different sample sizes in T(): large n problem 5.5 General testing rules for T(), using the severe testing concept The severe testing concept and confidence intervals 6.1 Dualities between one and two-sided intervals and tests 6.2 Avoiding shortcomings of confidence intervals Beyond the N–P paradigm: pure significance, and misspecification tests Concluding comments: have we shown severity to be a basic concept in a N–P philosophy of induction? (shrink)

_An engaging, accessible survey of the ethical issues faced by engineers, designed for students_ The first engineering ethics textbook to use debates as the framework for presenting engineering ethics topics, this engaging, accessible survey explores the most difficult and controversial issues that engineers face in daily practice. Written by a leading scholar in the field of engineering and computer ethics, Deborah Johnson approaches engineering ethics with three premises: that engineering is both a technical and a social endeavor; that engineers (...) don’t just build things, they build society; and that engineering is an inherently ethical enterprise. (shrink)

I document some of the main evidence showing that E. S. Pearson rejected the key features of the behavioral-decision philosophy that became associated with the Neyman-Pearson Theory of statistics (NPT). I argue that NPT principles arose not out of behavioral aims, where the concern is solely with behaving correctly sufficiently often in some long run, but out of the epistemological aim of learning about causes of experimental results (e.g., distinguishing genuine from spurious effects). The view Pearson did hold gives a (...) deeper understanding of NPT tests than their typical formulation as accept-reject routines, against which criticisms of NPT are really directed. The Pearsonian view that emerges suggests how NPT tests may avoid these criticisms while still retaining what is central to these methods: the control of error probabilities. (shrink)

The error statistical account of testing uses statistical considerations, not to provide a measure of probability of hypotheses, but to model patterns of irregularity that are useful for controlling, distinguishing, and learning from errors. The aim of this paper is (1) to explain the main points of contrast between the error statistical and the subjective Bayesian approach and (2) to elucidate the key errors that underlie the central objection raised by Colin Howson at our PSA 96 Symposium.

Floridi and Sanders, seminal work, “On the morality of artificial agents” has catalyzed attention around the moral status of computer systems that perform tasks for humans, effectively acting as “artificial agents.” Floridi and Sanders argue that the class of entities considered moral agents can be expanded to include computers if we adopt the appropriate level of abstraction. In this paper we argue that the move to distinguish levels of abstraction is far from decisive on this issue. We also argue that (...) adopting certain levels of abstraction out of context can be dangerous when the level of abstraction obscures the humans who constitute computer systems. We arrive at this critique of Floridi and Sanders by examining the debate over the moral status of computer systems using the notion of interpretive flexibility. We frame the debate as a struggle over the meaning and significance of computer systems that behave independently, and not as a debate about the ‘true’ status of autonomous systems. Our analysis leads to the conclusion that while levels of abstraction are useful for particular purposes, when it comes to agency and responsibility, computer systems should be conceptualized and identified in ways that keep them tethered to the humans who create and deploy them. (shrink)

A toast is due to one who slays Misguided followers of Bayes, And in their heart strikes fear and terror With probabilities of error! (E.L. Lehmann).

The concept of agency as applied to technological artifacts has become an object of heated debate in the context of AI research because some AI researchers ascribe to programs the type of agency traditionally associated with humans. Confusion about agency is at the root of misconceptions about the possibilities for future AI. We introduce the concept of a triadic agency that includes the causal agency of artifacts and the intentional agency of humans to better describe what happens in AI as (...) it functions in real-world contexts. We use the VW emission fraud case to explain triadic agency since in this case a technological artifact, namely software, was an essential part of the wrongdoing and the software might be said to have agency in the wrongdoing. We then extend the case to include futuristic AI, imagining AI that becomes more and more autonomous. (shrink)

In a recent discussion note Sober (1985) elaborates on the argument given in Sober (1982) to show the inadequacy of Ronald Giere's (1979, 1980) causal model for cases of frequency-dependent causation, and denies that Giere's (1984) response avoids the problem he raises. I argue that frequency-dependent effects do not pose a problem for Giere's original causal model, and that all parties in this dispute have been guity of misinterpreting the counterfactual populations involved in applying Giere's model.

Cartwright argues for being a realist about theoretical entities but non-realist about theoretical laws. Her reason is that while the former involves causal explanation, the latter involves theoretical explanation; and inferences to causes, unlike inferences to theories, can avoid the redundancy objection--that one cannot rule out alternatives that explain the phenomena equally well. I sketch Cartwright's argument for inferring the most probable cause, focusing on Perrin's inference to molecular collisions as the cause of Brownian motion. I argue that either the (...) inference she describes fails to be a genuinely causal one, or else it too is open to the redundancy objection. However, I claim there is a way to sustain Cartwright's main insight: that it is possible to avoid the redundancy objection in certain cases of causal inference from experiments (e.g., Perrin). But, contrary to Cartwright, I argue that in those cases one is able to infer causes only by inferring some theoretical laws about how they produce experimental effects. (shrink)

The growing availability of computer power and statistical software has greatly increased the ease with which practitioners apply statistical methods, but this has not been accompanied by attention to checking the assumptions on which these methods are based. At the same time, disagreements about inferences based on statistical research frequently revolve around whether the assumptions are actually met in the studies available, e.g., in psychology, ecology, biology, risk assessment. Philosophical scrutiny can help disentangle 'practical' problems of model validation, and conversely, (...) a methodology of statistical model validation can shed light on a number of issues of interest to philosophers of science. (shrink)

Responsible Robotics is about developing robots in ways that take their social implications into account, which includes conceptually framing robots and their role in the world accurately. We are now in the process of incorporating robots into our world and we are trying to figure out what to make of them and where to put them in our conceptual, physical, economic, legal, emotional and moral world. How humans think about robots, especially humanoid social robots, which elicit complex and sometimes disconcerting (...) reactions, is not predetermined. The animal–robot analogy is one of the most commonly used in attempting to frame interactions between humans and robots and it also tends to push in the direction of blurring the distinction between humans and machines. We argue that, despite some shared characteristics, when it comes to thinking about the moral status of humanoid robots, legal liability, and the impact of treatment of humanoid robots on how humans treat one another, analogies with animals are misleading. (shrink)

While many philosophers of science have accorded special evidential significance to tests whose results are "novel facts", there continues to be disagreement over both the definition of novelty and why it should matter. The view of novelty favored by Giere, Lakatos, Worrall and many others is that of use-novelty: An accordance between evidence e and hypothesis h provides a genuine test of h only if e is not used in h's construction. I argue that what lies behind the intuition that (...) novelty matters is the deeper intuition that severe tests matter. I set out a criterion of severity akin to the notion of a test's power in Neyman-Pearson statistics. I argue that tests which are use-novel may fail to be severe, and tests that are severe may fail to be use-novel. I discuss the 1919 eclipse data as a severe test of Einstein's law of gravity. (shrink)

The crisis of replication has led many to blame statistical significance tests for making it too easy to find impressive looking effects that do not replicate. However, the very fact it becomes difficult to replicate effects when features of the tests are tied down actually serves to vindicate statistical significance tests. While statistical significance tests, used correctly, serve to bound the probabilities of erroneous interpretations of data, this error control is nullified by data-dredging, multiple testing, and other biasing selection effects. (...) Arguments claiming to vitiate statistical significance tests attack straw person variants of tests that commit well-known fallacies and misinterpretations. There is a tension between popular calls for preregistration – arguably, one of the most promising ways to boost replication – and accounts that downplay error probabilities: Bayes Factors, Bayesian posteriors, likelihood ratios. By underscoring the importance of error control for well testedness, the replication crisis points to reformulating tests so as to avoid fallacies and report the extent of discrepancies that are and are not indicated with severity. (shrink)

After some general remarks about the interrelation between philosophical and statistical thinking, the discussion centres largely on significance tests. These are defined as the calculation of p-values rather than as formal procedures for ‘acceptance‘ and ‘rejection‘. A number of types of null hypothesis are described and a principle for evidential interpretation set out governing the implications of p- values in the specific circumstances of each application, as contrasted with a long-run interpretation. A number of more complicated situ- ations are discussed (...) in which modification of the simple p-value may be essential. (shrink)

In this paper I use the concept of forbidden knowledge to explore questions about putting limits on science. Science has generally been understood to seek and produce objective truth, and this understanding of science has grounded its claim to freedom of inquiry. What happens to decision making about science when this claim to objective, disinterested truth is rejected? There are two changes that must be made to update the idea of forbidden knowledge for modern science. The first is to shift (...) from presuming that decisions to constrain or even forbid knowledge can be made from a position of omniscience (perfect knowledge) to recognizing that such decisions made by human beings are made from a position of limited or partial knowledge. The second is to reject the idea that knowledge is objective and disinterested and accept that knowledge (even scientific knowledge) is interested. In particular, choices about what knowledge gets created are normative, value choices. When these two changes are made to the idea of forbidden knowledge, questions about limiting or forbidding lines of inquiry are shown to distract attention from the more important matters of who makes and how decisions are made about what knowledge is produced. Much more attention should be focused on choosing directions in science, and as this is done, the matter of whether constraints should be placed on science will fall into place. (shrink)

Chow correctly pinpoints several confusions in the criticisms of statistical hypothesis testing but his book is considerably weakened by its own confusions about concepts of testing (perhaps owing to an often very confusing literature). My focus is on his critique of power analysis (Ch. 6). Having denied that NHSTP considers alternative statistical hypotheses, and having been misled by a quotation from Cohen, Chow finds power analysis conceptually suspect.

The first topic of concern is anonymity, specifically the anonymity that is available in communications on the Internet. An earlier paper argues that anonymity in electronic communication is problematic because: it makes law enforcement difficult ; it frees individuals to behave in socially undesirable and harmful ways ; it diminishes the integrity of information since one can't be sure who information is coming from, whether it has been altered on the way, etc.; and all three of the above contribute to (...) an environment of diminished trust which is not conducive to certain uses of computer communication. Counterbalancing these problems are some important benefits. Anonymity can facilitate some socially desirable and beneficial behavior. For example, it can eliminate the fear of repercussions for behavior in contexts in which repercussions would diminish the availability or reliability of information, e.g., voting, personal relationships between consenting adults, and the like. Furthermore, anonymity can be used constructively to reduce the effect of prejudices on communications. Negative aspects of anonymity all seem to point to a tension between accountability and anonymity. They suggest that accountability and anonymity are not compatible, and they even seem to suggest that since accountability is a good thing, it would be good to eliminate anonymity. In other words, the problems with anonymity suggest that individuals are more likely to behave in socially desirable ways when they are held accountable for their behavior, and they are more likely to engage in socially undesirable behavior when they are not held accountable. I am not going to take issue with the correlation between accountability and anonymity, but rather with the claim that accountability is good. To examine this problem, let's look at a continuum that stretches from total anonymity at one end, and no anonymity at all at the other end. At the opposite extreme of anonymity is a panopticon society. The panopticon is the prison environment described by Foucault in which prison cells are arranged in a large circle with the side facing the inside of the circle open to view. The guard tower is placed in the middle of the circle so that guards can see everything that goes on in every cell. In a recent article on privacy, Jeffrey Reiman, reflecting on the new intelligent highway systems, suggests that we are moving closer and closer to a panopticon society. When we contemplate all the electronic data that is now gathered about each one of us as we move through our everyday lives- intelligent highway systems, consumer transactions, traffic patterns on the internet, medical records, financial records, and so on- we see the trend that Reiman identifies. Electronic behavior is recorded and the information is retained. While actions/transactions in separate domains are not necessarily combined, it seems obvious that the potential exists for combining data into a complete portfolio of an individual's day to day life. So it would seem that as more and more activities and domains are moved into a IT-based medium, the closer we will come to a panopticon society. A panopticon society gives us the ultimate in accountability. Everything an individual does is observable and therefore available to those to whom we are accountable. Of course, in doing this, it puts us, in effect, in prison. The prison parallel is appropriate here because what anonymity allows us is freedom; prison is the ultimate in lack of freedom. In this way the arguments for a free society become arguments for anonymity. Only when individuals are free will they experiment, try new ideas, take risks, and learn by doing so. Only in an environment that tolerates making mistakes will individuals develop the active habits that are so essential for democracy. In a world without information technology, individuals have levels or degrees and various kinds of anonymity and consequently different levels and kinds of freedom. Degrees and kinds of anonymity vary with the domain: small town social life versus urban social life, voting, commercial exchanges, banking, automobile travel, airplane travel, telephone communication, education, and so on. Drawing from our experience before IT-based institutions, we might believe that what we need is varying levels or degrees and kinds of anonymity. This seems a good starting place because it suggests an attempt to re-create the mixture that we have in the physical, non-IT-based world. Nevertheless, there is a danger. If we think in terms of levels and degrees of anonymity, we may not see the forest from the trees. We may not acknowledge that in an electronic medium, levels and kinds of anonymity mean, in an important sense, no anonymity. If there are domains in which we can be anonymous but those domains are part of a global communication infrastructure in which there is no anonymity at the entry point, then it will always be possible to trace someone's identity. We delude ourselves when we think we have anonymity on-line or off-line. Rather, what we have both places is situations in which it is more and less difficult to identify individuals. We have a continuum of situations in which it is easy and difficult to link behavior to other behavior and histories of behaviors. In the physical world, we can go places and do things where others don't know us by name and have no history with us, though they see our bodies, clothes, and behavior. If we do nothing unusual, we may be forgotten. On the other hand, if we do something illegal, authorities may attempt to track us down and figure out who we are. For example, law enforcement officials, collection agencies, those who want to sue us may take an active interest in removing our anonymity, ex post facto. Think of Timothy McVeigh and Terry Nichols-the men who apparently bombed the federal building in Oklahoma City. Much of what they did, they did anonymously, but then law enforcement officials set out to find out who had done various things, e.g., rented a car, bought explosives, etc. The shrouds of anonymity under which McVeigh and Nichols had acted were slowly removed. Is this any different than behavior on the internet? Is there a significant difference in the kind or degree of anonymity we have in the physical world versus what we have in an IT-based world? The character of the trail we leave is different; in the one case, its an electronic trail; while in the other it involves human memories, photographs, and paper and ink. What law enforcement officials had to do to track down McVeigh is quite different from tracking down an electronic law breaker. Also, the cost of electronic information gathering, both in time and money, can be dramatically lower than the cost of talking to people, gathering physical evidence, and the other minutia required by traditional detective work. We should acknowledge that we do not and are never likely to have anonymity on the Internet. We would do better to think of different levels or kinds of identity. There are important moral and social issues arising as a result of these varying degrees and kinds of identity. Perhaps the most important matter is assuring that individuals are informed about the conditions in which they are interacting. Perhaps, even more important is that individuals have a choice about the conditions under which they are communicating. In the rest of this paper we explore a few examples of levels and kinds of identity that are practical on the Internet. We discuss the advantages and disadvantages that we see for these "styles" of identity for individuals, and we examine the costs and benefits of these styles for society as a whole. (shrink)

Despite its widespread use in science, the Neyman-Pearson Theory of Statistics (NPT) has been rejected as inadequate by most philosophers of induction and statistics. They base their rejection largely upon what the author refers to as after-trial criticisms of NPT. Such criticisms attempt to show that NPT fails to provide an adequate analysis of specific inferences after the trial is made, and the data is known. In this paper, the key types of after-trial criticisms are considered and it is argued (...) that each fails to demonstrate the inadequacy of NPT because each is based on judging NPT on the grounds of a criterion that is fundamentally alien to NPT. As such, each may be seen to either misconstrue the aims of NPT, or to beg the question against it. (shrink)

The key problem in the controversy over group selection is that of defining a criterion of group selection that identifies a distinct causal process that is irreducible to the causal process of individual selection. We aim to clarify this problem and to formulate an adequate model of irreducible group selection. We distinguish two types of group selection models, labeling them type I and type II models. Type I models are invoked to explain differences among groups in their respective rates of (...) production of contained individuals. Type II models are invoked to explain differences among groups in their respective rates of production of distinct new groups. Taking Elliott Sober's model as an exemplar, we argue that although type I models have some biological importance--they force biologists to consider the role of group properties in influencing the fitness of organisms--they fail to identify a distinct group-level causal selection process. Type II models if properly framed, however, do identify a group-level causal selection process that is not reducible to individual selection. We propose such a type II model and apply it to some of the major candidates for group selection. (shrink)

We argue that a responsible analysis of today's evidence-based risk assessments and risk debates in biology demands a critical or metascientific scrutiny of the uncertainties, assumptions, and threats of error along the manifold steps in risk analysis. Without an accompanying methodological critique, neither sensitivity to social and ethical values, nor conceptual clarification alone, suffices. In this view, restricting the invitation for philosophical involvement to those wearing a "bioethicist" label precludes the vitally important role philosophers of science may be able to (...) play as bioevidentialists. The goal of this paper is to give a brief and partial sketch of how a metascientific scrutiny of risk evidence might work. (shrink)

In seeking general accounts of evidence, confirmation, or inference, philosophers have looked to logical relationships between evidence and hypotheses. Such logics of evidential relationship, whether hypothetico-deductive, Bayesian, or instantiationist fail to capture or be relevant to scientific practice. They require information that scientists do not generally have (e.g., an exhaustive set of hypotheses), while lacking slots within which to include considerations to which scientists regularly appeal (e.g., error probabilities). Building on my co-symposiasts contributions, I suggest some directions in which a (...) new and more adequate philosophy of evidence can move. (shrink)

A critically important ethical issue facing the AI research community is how AI research and AI products can be responsibly conceptualised and presented to the public. A good deal of fear and concern about uncontrollable AI is now being displayed in public discourse. Public understanding of AI is being shaped in a way that may ultimately impede AI research. The public discourse as well as discourse among AI researchers leads to at least two problems: a confusion about the notion of (...) ‘autonomy’ that induces people to attribute to machines something comparable to human autonomy, and a ‘sociotechnical blindness’ that hides the essential role played by humans at every stage of the design and deployment of an AI system. Here our purpose is to develop and use a language with the aim to reframe the discourse in AI and shed light on the real issues in the discipline. (shrink)

This chapter is reprinted from our lead essay in the Encyclopedia of Science, Technology, and Ethics, ed. C. Mitcham, Gale, 2005.

Although both philosophers and scientists are interested in how to obtain reliable knowledge in the face of error, there is a gap between their perspectives that has been an obstacle to progress. By means of a series of exchanges between the editors and leaders from the philosophy of science, statistics and economics, this volume offers a cumulative introduction connecting problems of traditional philosophy of science to problems of inference in statistical and empirical modelling practice. Philosophers of science and scientific practitioners (...) are challenged to reevaluate the assumptions of their own theories - philosophical or methodological. Practitioners may better appreciate the foundational issues around which their questions revolve and thereby become better 'applied philosophers'. Conversely, new avenues emerge for finally solving recalcitrant philosophical problems of induction, explanation and theory testing. (shrink)